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MUTATION AND CHAOS IN NONLINEAR MODELS OF HEREDITY 

NASIR GANIKHODJAEV, MANSOOR SABUROV, AND ASHRAF MOHAMED NAWI 

Abstract. In this short communication, we shall explore a nonlinear discrete dynamical system that 
. . , naturally occurs in population systems to describe a transmission of a trait from parents to their 

CO ' offspring. We consider a Mendelian inheritance for a single gene with three alleles and assume that to 

form a new generation, each gene has a possibility to mutate, that is, to change into a gene of the other 

kind. We investigate the derived models. A numerical simulation assists us to get some clear picture 

about chaotic behaviors of such models. 
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Q ■ 1. Introduction 

1-^ , Recently chaotic dynamical systems are very popular in science and engineering. Besides the original 

Cd ' definition of Li-Yorke chaos in [1], there have been various definitions for "chaos" in the literature, 

and the most often used one is given by Devaney in [^. Although there is no universal definition for 

chaos, the essential feature of chaos is sensitive dependence on initial conditions so that the eventual 

behavior of the dynamics is unpredictable. The theory and methods of chaotic dynamical systems have 

^ . been of fundamental importance not only in mathematical sciences, but also in physical, engineering, 

^^ I biological, and even economic sciences. In this paper, a chaos would be understood in the sense of 

" Li-Yorke (see [Sill!). 

ly^ i In this short communication, we introduce and examine a family of nonlinear discrete dynamical 

,-4^ ' systems that naturally occurs to describe a transmission of a trait from parents to their offspring. Here, 
(^ . we shall present some essential analytic and numerical results on dynamics of such models by avoiding 
f^ I some technical parts of the proofs. In upcoming papers, we shall explore the deepest investigation on 
asymptotic behaviors of the family of nonlinear dynamical systems by providing the proofs in details. 
As the first example, we consider a Mendelian inheritance of a single gene with two alleles A and a 
^ I (see [5j). Let an element x represent a gene pool for a population and it have been expressed a linear 
H ■ combination of the alleles A and a 

(1.1) X = xi A + 2;2a 

where, < xi,X2 < 1 and xi + X2 = 1. Then, xi,X2 are the percentage of the population which 
carries the alleles A and a respectively. The rules of the Mendelian inheritance indicate that the next 
generation 

x' = x'x A + X2a 
represents a gene pool for the population which carries the alleles A and a respectively, where 

.. 2^ J^l = PaA,aA + '^PAa,AXlX2 + Paa,Axl 

I ^2 = J~^AA,aXi + 2PAa,aXlX2 + ^aa,a2^2 

Here, Paa,a (resp. PAA,a) is the probability that the child receives the allele A (resp. a) from 
parents with the allele A; PAa,A (resp. PAa,a) is the probability that the child receives the allele A 
(resp. a) from parents with the alleles A and a respectively; and Paa,A (resp. Paa,a) is the probability 

1 
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that the child receives the allele A (resp. a) from parents with allele a. It is evident that 

P.., A + P.., a = 1) PAa,A = PaA,A, PAa,a = PaA,a, X]^ + 2:2 = 1- 

Thus, (|1.2p is a nonlinear dynamical system acting on the one dimensional symplex 

5^ = {(xi,X2) GM^ : j;i,X2 >0,xi+X2 = l} 

that describes the distribution of the next generation which carries the alleles A and a respectively, if 
the distribution of the current generation are known. 

Recall that in the Mendelian inheritance case, i.e., Paa,a = Paa,a = 1 and PAA,a = Paa,A = 0, the 
dynamical system (jl.2p has the following form 

.. „x \x[ = Xi + 2PAa,AXlX2 

|x'2 = 2PAa,aXlX2 + xj. 

We assume that prior to a formation of a new generation each gene has a possibility to mutate, that 
is, to change into a gene of the other kind. Specifically, we assume that for each gene the mutation 
A — )• a occurs with probability a, and a — > A occurs with probability /3. We assume that " the mutation 
occurs if only if both parents have the same allele''\ Then, we have that PAA,a = O) Paa,A = /3, 
Paa,a = 1 — a, Paa,a = 1-/3 and the dynamical system (|1.2p has the following form 



'1 = (1 - a)xl + 2PAa,A^iX2 + (3x1 

ax\ + 2PAa,aX\X2 + (1 - ^)x\. 
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An operator V : S^ ^ S^ given by (jl.4p is called a quadratic stochastic operator. 

We introduce some standard terms in the theory of a discrete dynamical system V : X ^ X. 

A sequence {x(")}^q is called a trajectory of V starting from an initial point x", where x'*^-* = 
y^-jj-(n-i)^ for any n G N. Recall that a point x is called a fixed point of V if V{x.) = x. We denote 
a set of all fixed points by Fix(y). A dynamical system V is called regular if a trajectory {x^^^jJ^q 
converges for any initial point x. Note that if V is regular, then the limiting point is a fixed point of V. 
Thus, in a regular system, the fixed point of dynamical system V describes a long run behavior of the 
trajectory of V starting from any initial point. The biological treatment of the regularity of dynamical 
system V is rather clear: in a long run time, the distribution of species in the next generation coincide 
with distribution of species in the current generation, i.e., stable. 

A fixed point set and an omega limiting set of quadratic stochastic operators (QSO) were deeply 
studied in [6]- [TO], and they play an important role in many applied problems pT|fl2]. In the paper 
[13j . it was given a long self-contained exposition of recent achievements and open problems in the 
theory of quadratic stochastic operators. 

Definition 1.1. A dynamical system V : X ^ X is said to be ergodic if the limit 

n-l 



(1.5) lim -YvHx) 

exists for any x £ X. 



n 

k=0 



Based on some numerical calculations, S.Ulam has conjectured [T^ that any QSO V : S"^^^ — )> 
S"™"^ acting on the finite dimensional simplex is ergodic. However, in [15], M.Zakharevich showed 
that, in general, Ulam's conjecture is false. Namely, M.Zakharevich showed that the following QSO 
Vq : S"^ ^- S'^ is not ergodic 

(1.6) Vo 



x[ 


= xf + 2xiX2 


x'. 


= X2 + 2X2X3 


4 


= x| + 2x3X1 
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In |16] . Zakharevich's result was generalized in the class of Volterra QSO. Moreover, in [16], it was 
given a necessary and sufficient condition of being non-ergodicity of Volterra QSO defined on 5^. 
We define the fe-th order Cesaro mean by the following formula 

j=0 

where k > 1 and Ces^ (x, V) = y"(x). It is clear that the first order Cesaro mean Ces]^ (^j ^) 

n-l 

is nothing but - ^ V^{x.). In this manner, Zakharevich's result says that the first order Cesaro 
" fc=o 

Ces\ (x, Vq) > of the trajectory of operator (jl.6p diverges for any initial point except fixed 

J n=0 

Cesl. (x, Vo) f ) where 

J n=0 

/c > 1, of the trajectory of operator (jl.6p diverges for any initial point except fixed points. This leads 
to a conclusion that the operator Vq given by (jl.6p might have unpredictable behavior. 

In fact, in [18], it was proven that an operator Vq given by (jl.6p exhibits the Li-Yorke chaos. It is 
worth pointing out that some other properties of Volterra QSO were studied in [T9|[20]. 

Note that if QSO is regular, then it is ergodic. However, the reverse implication is not always true. 
It is known that the dynamical system (jl.4p is either regular or converges to a periodic-2 point (see 
[21j). Therefore, in ID simplex, any QSO is ergodic. In other words, a mutation in population system 
having a single gene with two alleles always exhibits an ergodic behavior (or almost regular). It is 
of independent interest to study a mutation in population system having a single gene with three 
alleles. In the next section, we consider an inheritance of a single gene with three alleles ai,a2 and 
as and show that a nonlinear dynamical system corresponding to the mutation exhibits a non-ergodic 
behavior (or Li-Yorke chaos). 

2. Inheritance for a Single Gene with Three Alleles 

In this section, we shall derive a mathematical model of a inheritance of a single gene with three 
alleles. In this case, an element x represents a population if its expression x = xioi + ^202 + x^a^, as 
a linear combination of the alleles ai,a2 and a^, satisfies the following conditions < a;i,a;2,a:;3 < 1 
and xi + X2 + xa = 1. Then xi,X2,X3 are the percentage of the population which carries the alleles 
ai , a2 and a^ respectively. 

We assume that prior to a formation of a new generation each gene has a possibility to mutate, 
that is, to change into a gene of the other kind. We assume that the mutation occurs if both parents 
have the same alleles. Specifically, we will consider two types of the simplest mutations: 

(1) Assume that mutations oi — )• a2, 02 — >• 03, and as — > ai occur with probability a; 

(2) Assume that mutations ai — )■ 03, 02 — )• oi, and 03 — > 02 occur with probability a. 
Then the corresponding dynamical systems are defined on the two-dimensional simplex 

5^ = {(a^i, 3^2, X3) eM.^ : X > 0, X2 > 0, 3:3 > 0, xi+ X2 + X3 = 1}. 
In the first mutation, we have 

(2.1) 

In the second mutation, we have 

(2.2) 





Xi - 


= (1- 


- a)x\ + 2a;ia;2 -|- ax"^ 


Va-. < 


Xn ' 


= (1- 


- a)x\ + 2j;23^3 + ctx\ 


1 i^'\rf^ 


A- 


= (1- 


- a)x1 + 2j;3j;i -|- ax\ 


LldVt: 


'1 


= (1- 


- a)x\ + 2xiX2 -I- ax\ 


W^:< 


Xn 


= (1- 


- a)x2 + 2x2X3 -I- axf 




A 


= (1- 


- a)x'^ + 2x3X1 -|- ax2 
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In both cases, if a = 0, i.e., if a mutation does not occur, then a dynamical systems ()2.ip and ()2.2p 
coincide with Zakharevich's operator (|1.6p . As we already mentioned, Zakharevich's operator exhibits 
a Li-Yorke chaos (see [IE]). 

Let a = 1. In the first case, an operator Vi is a permutation of Zakharevich's operator ()1.6p . An 
omega limiting set of the permutation of Zakharevich's operator was studied in [22]. By means of 
results pJj, [17], [E], and ^22j, one can show that the operator Vi is non-ergodic as well as a Li-Yorke 
chaos. Let a = 1. In the second case, an operator Wi is a permutation of an operator which was 
studied in [23] . By applying the same method which was given in [23] , we may show that the operator 
Wi is regular. 

It is easy to check that V^ = (1 — a)Vo + aVi and Wa = (1 — a)Wo + aWi. In other words, in the 
first case, a mutation operator is a convex combination of two Li-Yorke chaotic operators and in the 
second case, a mutation operator is a convex combination of a Li-Yorke chaotic and regular operators. 
Both operators Va and Wa were not studied in either paper [22] or [23]. In the next section, we are 
going to present some essential analytic and numerical results on dynamics of Va and Wa, by avoiding 
some technical parts of the proofs. In upcoming papers, we shall explore the deepest investigation on 
asymptotic behaviors of Va and Wa by providing the proofs in details. 



3. Attractors: Analytic and Numerical Results 



In this section, we shall study dynamics of the operators ()2.ip and ()2.2p . Here, we shall present 
some pictures of attractors of the operators (j2.ip and (j2.2p . 



3.1. Analytic Results on Dynamics of Va- We are aiming to present some analytic results on 
dynamics of Va : S'^ ^ S"^: 



(3.1) 



Va 



(1 — a)xf + 2xij;2 + 0X2 
(1 — a)x2 + 2x2X3 + axg 
(1 — a)x'^ + 2x3X1 -|- axf 



where Va{x) = x' = {x'l, X2, x'^) and < a < 1. As we already mentioned, this operator can be written 
in the following form: V^ = (1 — a)Vo + aVi for any < a < 1, where 



Vn 



x\ + 2xiX2 

xl + 2x2X3 

Xo + 2X3X1 



Let 



P 





X 


Vi: < 


X 




X 


1 \ 




1 
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X2 + 2xiX2 

x| + 2x2X3 
Xl + 2x3X1 



be a permutation. The proofs of the following results are straightforward. 

Proposition 3.1. Let V^ : 5^ — )■ 5^ he a quadratic stochastic operator given by (|3.ip . where a G (0, 1). 
Let Fix{Va) and uj{x^) be sets of fixed points and omega limiting points ofVa, respectively. Then the 
following statements hold true. 

(i) Operators P and Va are commutative, i.e., P oVa = Va° P; 

(ii) //x G FixiVa) then Px € Fix{Va); 

(iii) If FixiVa) is a finite set then \Fix{Va)\ = 1 {mod 3); 

(iv) One has that P{oj{x^)) = uj{Px^), for any x^ €5^. 

We are aiming to study the fixed point set FixiVa), where a € (0,1). It is worth mentioning 
that Fix{Vo) = {ei, 62, 63, C} and Fix{Vi) = {C}, where ei, 62, 63 are vertexes of the simplex S'^ and 
C 



^3' 3' 3 



3) is a center of the simplex S"^. 
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Recall [12] that a fixed point x^ G Fix{Va) is non-degenerate if and only if the following determinant 
is nonzero at the fixed point x": 



(3.2) 



8x'^ -. dx[ (9xj 

8x1 8x2 8x^ 

dx'2 dx'2 -, 8x2 

8x1 8x2 8x3 



1 



1 



1 



/o 



Proposition 3.2. Lei V^ : 5"^ — )• 5^ 6e a quadratic stochastic operator given by (j3.ip . where a S (0, 1). 
Zei C = (i, i, i) 6e a center of the simplex S"^. Then the following statements hold true: 

(i) All fixed points are non- degenerate; 
(ii) One has that Fix{Va) = {C}. 

Proof, (i). Let x € FixiVa) be a fixed point. One can easily check that 



4(1 — a + a'^){xiX2 + xix^ + X2X^) + 2a — 1 



If 2 ^ CK < 1 then the above expression is positive. This means that, in the case 2 ^ Q^ < !> all 
fixed point are non-degenerate. 

Let us consider the case < a < ^. In this case, the above expression is equal to zero if and only if 

a;iX2 + 3:1X3 + X2X3 



l< 1 

8x1 
dx'2 


8x'^ 


8x', 


8x2 
dx'y ^ 

8x2 1 


dx^ 
8x2 


8x1 


8x3 


1 


1 


1 



4(l-aW) • ^™^^ Xi -F X2 + X3 = 1, we have that xf -h x| -h x| = 2{l-a+a^) ■ 



Without loss of generality, we may assume that xi > max{x2,X3} (See Proposition 13.11 (i)). 

Let X2 > X3. Since x G Fix{Va), we have that X2 = (1 — a)x2 + 2x2X3 -|- 0X3. We then obtain that 

l-h2a2 



2(1 -a + a2) 



2 , 2 , 2 

Xt -p X2 \ Xn 



= xl + [(1 - a){xl + 2x2x3) + a(xi + 2x2x3)]^ + x^ 
< xl + [(1 - a)x2 + ax^f + X3 



\ X-j ~r ^2 ' "^S 



l + 2a^ 



2(1 -a + a"^) 



This is a contradiction. In the similar manner, one can have a contradiction whenever X3 > X2. This 
shows that, in the case < a < 2, all fixed points are non-degenerate. 

(ii). We want to show that Fix{Va) = {C}. It is clear that VaidS"^) C intS'^. This means 
that the operator Va does not have any fixed point on the boundary dS'^ of the simplex S'^, i.e., 
Fix{Va) n dS'^ = 0. Moreover, all fixed point are non-degenerate. Due to Theorem 8.1.4 in |12j . 
|Fix(yQ,)| should be odd. On the other hand, due to Corollary 8.1.7 in [12], one has that |Fix(FQ,)| < 4. 
Proposition 13.11 (iii) yields that |Fix(yQ,)| = 1. Simple calculations show that Fix{Va) = {C}. D 

We can easily check a local behavior of the fixed point C = (3,3,3). 

Proposition 3.3. Let Va '■ S"^ ^ S'^ be a quadratic stochastic operator given by ()3.ip . where a S (0, 1). 
Then the following statements hold true: 

(i) If a ^ ^ then the fixed point C = (5, -5, -3) is repelling; 
(ii) If a = 2 then the fixed point C = (3, 3, 3) is non-hyperbolic. 

We shall separately study two cases a 7^ 2 ^^"^ ^ — 2- 

Theorem 3.4. Let Vq, : S"^ — > 5^ he a quadratic stochastic operator given by ()3.ip . where a ^ ^. 
Then uj{x^) C intS'^ is an infinite compact subset of intS"^ for any x^ 7^ C . 
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Proof. Let a / ^. Since Va is continuous and Va^S"^) C intS"^, an omega limiting set uj{x^) is a 
nonempty compact set and uj{x^) C intS'^, for any x^ ^ C. We want to show that cj(j;'^) is infinite for 
any x^ ^ C. Since C is repehing, we have that C ^ u}{x^). Let us pick up any point x* € uj{x^) from 

Va (x* ) > 

J n=l 

of the point x* is infinite. Since Va is continuous, we have that < Va (x*) > C uj{x^). This shows 

I J n=l 

that a;(x'^) is infinite for any x^ ^ C. D 

Remark 3.5. It is worth mentioning that the sets of omega limiting points wvb {^^) '^''^d uvi i^^) of both 
operators Vq and Vi are infinite. However, unlike the operator Va, we have inclusions covoi^^) C dS^ 
and uviix^) C dS'^. Moreover, both operators Vq and Vi are non-ergodic. 

Conjecture 3.6. Let Va '■ S"^ ^ 5^ be a quadratic stochastic operator given by (j3.ip . where a ^ ^. 
Then the following statements hold true: 

(i) The operator Va is non-ergodic; 
(ii) The operator Va exhibits a Li-Yorke chaos. 

Now, we shall study the case a = 2- The operator 1^/2 : S^ — )• 5^ takes the following form 

Xi ^ ^^1 ' ^'^1*^2 ' ^"^2 
(3.3) ^1/2 ■ 1x2 = \xl + 2X2X3 + ix^ 

Xo ^ ^7^3 ~r ^X3Xl ~r ^Xi 

In this case, the fixed point C = (3,3)3) is non-hyperbolic and the spectrum of the Jacobian of 
the operator 1^1/2 at the fixed point C is Sp{J{C)) = |2, ^^f^j. 
Let us define the following sets 

Zi = {x G S^ : X2 = X3}, ^2 = {x G 5^ : xi = X3}, ^3 = {x G S*^ : xi = X2}, 
5i = {x G S*^ : xi > X2 > X3}, 5*2 = {x G S"^ : xi > X3 > X2}, 
5*3 = {x G S*^ : X3 > xi > X2}, 54 = {x G 5^ : X3 > X2 > xi}, 
55 = {x G S*^ : X2 > X3 > xi}, ^6 = {x G S^ : X2 > xi > X3}. 

Proposition 3.7. We have the following cycles: 
(1) h — yh — ^ ^3 — > h; 

(nj Di > D2 ?• '-'3 ?• '-'4 ^ '-'5 ^ '-'6 ^ '-'ij 

Proof. Let V1/2 be an operator given by (|3.3|) . One can easily check that 

^1 -X2 = [Xl - X3) ^ 

(3-4) x'l - X3 = (x2 - X3) 

X2-X3 = (X2-X1) — ^ — 

The proof the proposition follows from the above equality. D 

Theorem 3.8. Let V^i^ : S'^ ^ S'^ be a quadratic stochastic operator given by (j3.3p . The following 
statements hold true: 

(i) 4>{x) = \xi — X2||xi — X3IIX2 — X3I is a Lyapunov function; 
(ii) The trajectory always converges to the fixed point C = (|, |, ^). 
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Proof, (i). Let yi/2 be an operator given by (|3.3p . It follows from (j3.4p that 

,.-,. , -.. ., 1 + 3x1 1 + 3x2 1 + 3x3 

(t>{Vl/2{x)) = (/'(x) ^ ^ ^ . 

On the other hand, we have that 



1 + 3xi 1 + 3x2 1 + 3x3 



1+3x1 



< 



1+3x2 I l+3x'3 



1. 



Therefore, one has that (t){y{x)) < (j){x) for any x € S'^. This means that (p is decreasing a long the 
trajectory of Vi/2- Consequently, is a Lyapunov function. 

(ii). We know that wl^irtH^)) f is a decreasing bounded sequence. Therefore, the limit 
lim (j) (V-. /J{x)] = A exists. We want to show that A = 0. Suppose that A 7^ 0. It means that 

{^V2 (^)}°° ^ -^^ \ i^i U ^2 U /3}- Since A y^ 0, we get that 



lim ■ 

n— >-oo 



'1/2 



Vr,7'>ix) 



(n). 



<^K;;^(x) 



lim 

n—^oo 



•l + 3^W 1 + 34") 1 + 34")' 



On the other hand, since < V-^^^{x) > C S*^ \ {/i U ^2 U ^3}, there exists Eq such that for any n one 

L ' J n=l 

has that 



1 + 3^(") 1 + 3^(-) 1 + 3^(") 



< 1 - £0- 



2 2 2 

This is a contradiction. It shows that A = 0. 
Therefore, w(x°) C /i U ^2 U ^3. We want to show that a;(x'^) = Zi fl /2 H ^3. 



We know that 



(") ^(") 



Xn 



max- 



(n) (n) 



.(«) 



X q 



X. 



(n) 



(") _ ™(") 
2 ■''3 

J") _ Jn) 
l-^ X3 



0. It follows from (lOD that 



„(") ^{«) 



}.. 



0. 



This means that ( x^ ,x 



(") ^(") ^(")^ ""^9° /'111 



1^2 , X3 



0,0,0). This completes the proof. 



D 



3.2. Numerical Results on Dynamics of V^. We are going to present some pictures of attractors 
(an omega limiting set) of the operator Va '■ S'^ ^ S^ given by (j3.ip . 

In the cases a = and a = 1, the operators Vq and Vi have similar behaviors. The trajectories of 
both operators, Vq and Vi, look as spirals, but one of them is moving by clockwise and another one is 
moving by anticlockwise. In these cases, we have that covoix^) C dS^ and u;vi(x°) C dS^. 

We are interested in the dynamics of the mutation operator Va while a approaches to ^ from both 
left and right sides. In order to see some symmetry, we shall provide attractors of Va and 14-0 at the 
same time. 

If we slightly change a from and 1, then we can see that the omega limiting set splits from the 
boundary. Moreover, in the picture, we can see "three vertexes" (it is roughly speaking) where 
the trajectory shall spend almost all its time around those points. Therefore, we are expecting non- 
ergodicity of the operator V^ if a is near by or 1. 

If a becomes close to the ^ then we can see some chaotic pictures. We observe from the pictures that, 
in the cases a and 1 — a, the attractors are the same but different from the orientations. One of them is 
moving by clockwise and another one is by anticlockwise. If a is enough close to ^ then we detect com- 
pletely different pictures. Here are some pictures for the values of a = 0.4995,0.4999,0.5005,0.5001. 
In this cases, attractors are disconnected sets and they consist of 6 connected sets. It is very surprising 
that why the number of connected sets are 6 and why not 5 or 7. 
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Figure 1. Attractors of T4: a = 0.1 and a = 0.9 





31 32 33 0.34 33 



0.51 0.52 33 34 3J 



Figure 2. Attractors of Va- a = 0.497 and a = 0.503 





31 ' 0^2 ' 0^3 ' oii ' 035 ' Cl5S 



31 32 33 34 33 3^ 



Figure 3. Attractors of Va- a = 0.499 and a = 0.501 




0.31 0.22 0.33 0.34 0.3i 




31 32 33 0.34 3J 



Figure 4. Attractors of K: a = 0.4995 and a = 0.5005 



For the operator (j3.ip . the fluctuation point is a = g- In this case, the influence of the chaotic 
operators Vq and Vi are the same. Therefore, the operator V0.5 becomes regular and ergodic. This 
completes the numerical study of the operator (j3.ip . 
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31 32 33 34 33 



31 32 33 34 35 3^ 



Figure 5. Attractors of V„: a = 0.4999 and a = 0.5001 



3.3. Analytic Results on Dynamics of Wa- We are aiming to present some analytic results on 
dynamics of Wa : S'^ ^ S'^: 



(3.5) 



Wa-.i 



x'l = (1 — a)xi + 2x1X2 + 0x3 
X2 = (1 — a)x| + 2x2X3 + axf 
X3 = (1 — a)x| + 2x3X1 + 0X2 



where Wa{x) = x' = (x'^,X2,X3) and < a < 1. As we already mentioned, this operator can be 
written in the following form: Wa = (1 — a)Wo + ocWi for any < a < 1, where 



^1 = a^i + 2xiX2 



Wn 



X2 



X2 + 2x2x3 

Xq + 2X3X1 



W^ 



X2 



x\ + 2xiX2 

X3 + 2X2X3 
x\ + 2x3x1 



As we already stated, the operator Wq = Vq is Zakharevich's operator ()1.6p and the operator l^i 
is a permutation of the operator which was studied in [23] . By means of methods which were used in 
[23], we can easily prove the following result. 

Proposition 3.9. Let W\ : S'^ ^f S'^ he a quadratic stochastic operator given by ()3.5p with a = 1. 
Then the following statements hold true: 

(i) The operator Wi has a unique fixed point C = (3, 3, 3) which is attracting; 
(ii) The vertexes of the simplex 61,62,63 are 3-periodic points; 



m 



x^ + X2 + x| — 3 is a Lyupanov function; 



(iv) The operator Wi is regular in intS . 

By means of the same methods and techniques which are used for the operator V^, we can prove 
the following results 

Proposition 3.10. Let W^ : S'^ 

'ill 

^3' 3' 3 

(ii) //I — ■^<a<l then the fixed point is attracting; 



S he a quadratic stochastic operator given hy (j3.5p . Then it has 
a unique fixed point C = (i, i, i), i.e., Fix{Wa) = {C}. Moreover, one has that: 

(i) // < a < 1 — -^ then the fixed point is repelling; 



(iii) If a = 1 — ^ then the fixed point is non-hyperbolic. 



Theorem 3.11. Let Wa : 5^ 

following statements hold true: 



S be a quadratic stochastic operator given by (j3.5p . Then the 



(i) //O < a < 1 — -^ then lo{x^) C intS"^ is an infinite compact set for any x^ 7^ C; 
(h) // 1 - ^ < a < 1 then a;(x°) = {C} for any x° € 5^. 
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Conjecture 3.12. Let Wa : S"^ ^ S"^ be a quadratic stochastic operator given by (j3.5p . where < 
a < 1 — ■^. Then the following statements hold true: 

(i) The operator Wa is non-ergodic; 
(ii) The operator Wa exhibits a Li- Yorke chaos. 

3.4. Numerical Results on Dynamics of Wa- We are going to present some pictures of attractors 
(an omega limiting set) of the operator Wa : 5^ — )• 5^ given by (13. Sp . 

In this cases a = and a = 1, the operator Wq is a chaotic operator and the operator Wi is regular. 
Since Wa = (1 — a)Tyo + aWi, the mutation operator Wa gives a transition from the chaotic behavior 
to the regular behavior. Consequently, we are aiming to find fluctuation points in which we can see 
the transition from the chaotic behavior to the regular behavior. 





Figure 6. Attractors of Wq,: a = 0.001 and a = 0.01 

If Q is very close to then attractors of the operator Wa are splitted from the boundary of the 
simplex. However, the influence of the operator Wq is still very high. Therefore, we are expecting that 
the operator Wa is non-ergodic and chaotic, whenever a is very close to 0. 

If a becomes to close to 0.13333 then we can see a different picture. In attractors, we are able to 
detect some "growing hairs" (it is a just terminology). Moreover, if we shall continue to increasing 
a then these "hairs" will start to rise and they eventually become straight. Therefore, a = 0.13333 is 
the first fluctuation point. It is very interesting that the number of "hairs" is equal to 12. It is 
again a question: why do we have 12 numbers of "hears" and why not 11 or 13. 




31 33 33 34 35 36 




31 32 D33 34 35 



Figure 7. Attractors of Wa'. a = 0.13333 and a = 0.1338 



From these pictures, we can find another fluctuation point a = 1 — -^ ~ 0.1339745. Therefore, 
in order to have a transition from a chaotic behavior to the regular behavior, we should cross from 
two fluctuation points a = 0.13333 and a = 1 - ^ ~ 0.1339745. 
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Figure 8. Attractors of W„: a = 0.139, a = 0.144, and a = 0.150 



4. Conclusion 

In this paper, we present a mathematical model of the mutation in the biological environment 
having 3 alleles. We have presented two types of mutations. We have shown that a mutation (a 
mixing) in the system can be considered as a convex combination of Mendelian inheritances (extreme 
or non-mixing systems). The first mutation is a convex combination of two Li-Yorke chaotic systems 
and the second mutation is a convex combination of Li-Yorke chaotic and regular systems. 

In the first case, the first mutation can be considered as an evolution process between two different 
chaotic biological systems. In this case, we have shown that there is one fluctuation point (transition 
point) in order to change one chaotic system into another chaotic system. In the second case, the 
second mutation can be considered as an evolution process between chaotic and regular biological 
systems. In this case, we need two fluctuation points (transition points) in order to change the chaotic 
system into the regular system. We hope that we can find a similar phenomenon in nature. 
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